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Utilization  as  a  Dependent  Variable  in  MIS  Research 
Andrew  W.  Trice  and  Michael  E.  Treacy 

Abstract 

Utilization  of  an  information  system  is  an  important  and  frequently 
measured  MIS  variable,  since  use  of  a  system  is  the  conduit  through 
which  information  technology  can  affect  performance.  This  paper 
addresses  measurement  issues  associated  with  utilization  variables.  The 
discussion  is  motivated  by  a  review  of  the  literature  and  an  examination 
of  three  relevant  reference  theories.  It  is  asserted  that  utilization  can  be 
measured  effectively  if  the  measures  chosen  correspond  to  the  measures 
suggested  by  an  appropriate  reference  theory. 


Utilization  as  a  Dependent  Variable  in  MIS  Research 


I.  Introduction 

The  amount  of  use  an  individual,  group,  or  organization  makes  of  an  information 
system  is  a  key  variable  in  MIS  research.  It  is  often  used  as  an  independent  variable 
when  studying  or  predicting  the  impacts  that  an  information  system  has  had  on 
process,  structure,  and  performance.  The  degree  and  type  of  impacts  would  quite 
naturally  be  expected  to  vary  with  the  amount  of  use  that  is  made  of  the  system. 
Utilization  of  a  system  has  also  been  used  as  a  dependent  variable.  It  has  been 
modeled  as  an  outcome  construct  that  can  be  influenced  by  the  process  of  design  and 
implementation  and  by  characteristics  of  the  information  system,  the  task,  the 
individual  user  and  their  interaction.  Utilization  measures  are  also  of  great  practical 
significance  in  a  computing  environment  that  is  increasingly  driven  by  voluntary 
users. 

For  such  an  important  MIS  variable  as  information  system  use,  which  has  many 
readily  obtainable  measures,  it  is  somewhat  surprising  that  the  field  does  not  have 
generally  accepted  measurement  instruments.  The  lack  of  such  instruments  makes 
the  conduct  of  research  in  this  area  more  difficult  and  time  consuming.  It  also  means 
that  many  measures  are  quickly  developed  and  inadequately  validated.  The  wide 
variety  of  measures  that  are  now  employed  serves  to  slow  the  development  of  an 
accumulated  body  of  knowledge  on  the  factors  affecting  system  use  and  the  impacts 
of  system  use  on  elements  of  the  organization. 

Before  any  movement  can  be  made  toward  standard  instruments  for  measuring 
utilization  of  an  information  system,  there  are  important  and  difficult  conceptual 
problems  that  must  be  resolved.  Primary  among  these  is  deciding  what  aspect  of  use 
to  measure.  For  example,  an  individual's  use  of  spreadsheet  software  can  be 
characterized  in  several  different  ways:  by  the  time  spent,  functions  used,  or  models 
produced,  to  name  just  three.  The  appropriate  selection  should  be  guided  in  part  by 
the  purpose  which  the  measures  must  serve. 


This  paper  addresses  itself  to  some  of  these  conceptual  problems  of  measuring 
utilization.  It  begins  with  a  review  of  the  past  ten  years  of  research  literature  on 
factors  that  affect  the  use  of  information  systems.  From  this  literature  we  gain 
insights  into  the  state  of  practice  of  utilization  measurement,  some  of  the  conceptual 
problems  in  this  area,  and  the  needs  that  this  varied  research  has  for  different 
measures.  The  paper  then  turns  to  a  discussion  of  the  relevant  reference  theories  to 
which  utilization  research  has  turned  for  theoretical  support.  The  implications  of 
this  literature  for  utilization  measurement  are  examined  and  we  conclude  with  a 
discussion  of  needs  for  further  research. 

II.  Data  Base  of  Relevant  Articles,  1975-1985 

A  total  of  17  articles  relating  various  factors  to  information  systems  utilization  were 
found  by  searching  10  journals  over  the  years  1975-1985.  These  were  felt  to  be 
representative  in  terms  of  methodologies,  variable  definitions  and 
operationalizations,  and  findings,  and  were  used  for  analysis.  The  Appendix 
contains  a  listing  of  these  articles. 

Empirical  studies  include  laboratory  experiments,  quasi-experiments,  and  case 
studies.  Of  these,  laboratory  experiments  are  encountered  the  least  frequently  in 
the  literature. 

III.  Survey  of  Utilization  Research 

The  vast  majority  of  research  in  the  utilization  area  implicitly  defines  utilization  as 
either  the  amount  o^ effort  expended  interacting  with  an  information  system  or,  less 
frequently,  as  the  number  of  reports  or  other  information  products  generated  by  the 
information  system  per  unit  time.  Examples  include  frequency  and  number  of 
computer  sessions,  connect  time,  time  spent  using  different  system  functions, 
number  of  records  updated,  and  keystrokes  or  carriage  returns,  i 


iThereare  two  small  bodies  of  utilization  research  whose  goals  are  fundamentally  difTerent  from  those  of  the  studies  which 
use  effort  or  information  product  as  a  dependent  variable  One  of  these  groups  of  studies  attempts  to  predict  the  rate  of 
acceptance  of  a  new  technology  (i.e.  microcomputers!.  In  these  studies,  utilization  is  defined  as  the  number  of  terminals, 
microcomputers,  printers,  or  other  l/S  component  currently  used  by  the  organization.  There  were  two  such  articles  found  in 
the  literature.  Randies  ( 1983)  and  Ilan  and  Shapira  (1985 1  both  developed  diffusion  models  to  predict  the  acceptance  rate  of  a 
new  technology  over  time.  These  diffusion  models  are  straightforward  applications  of  well-established  theories  drawn  from 
new  product  growth  models  in  marketing  (eg  Mahajan  and  Muller,  1979i  and  earlier,  the  Mansfield  model  of  imitation  rates 
of  technical  change  in  economics  (Mansfield,  1961 1  The  other  sroupofstudiesusesmonetary  measures  to  examine  empirical 
relationships  between  amount  of  computer  system  usage  and  firm  characteristics.  These  studies  used  dollar  figures 
(GremiUion,  1984)  or  DP  expenditure  proportions  (Turner,  1982;  Delone,  1981 )  to  approximate  degree  of  MIS  usage. 


The  Mason-Mitroff  (1973)  research  framework  provided  the  impetus  for  a  large 
stream  of  empirical  research  that  has  studied  the  relationship  between  individual 
utilization  of  an  information  system  and  four  general  types  of  independent  variables. 
This  research  structure  is  illustrated  in  Figure  1.  The  four  types  of  independent 
variables  are  design  and  implementation  process  variables,  information  system 
characteristics,  individual  differences,  and  task  characteristics. 
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Figure  1.  Structure  of  Utilization  Research 


Design  and  implementation  process  variables  refer  to  the  components  of  the  process 
of  introducing  information  technology  into  an  organization.  Examples  of  these 
variables  studied  include  amount  and  quality  of  training  (Schewe,  1976;  Fuerst  and 
Cheney,  1982),  overall  implementation  strategy  (Gremillion,  1980),  accuracy  of  user 
expectations  (Ginzberg,  1981),  support  of  top  management  (Schewe,  1976;  Fuerst 
and  Cheney,  1982;  Robey,  1979;  Raymond,  1985),  user  involvement  (Schewe,  1976; 
Fuerst  and  Cheney,  1982;  Mann  and  Watson,  1984),  understanding  of  the  task 


activities  of  potential  users  (Nichols,  1981),  and  sanctionary  power  and  presence  of  a 
third  party  (De  Brabander  and  Thiers,  1984).  In  some  cases,  implementation  was 
operationalized  more  or  less  as  a  binary  variable  (e.g.,  Crawford,  1982).  In  the 
context  of  implementation,  utilization  is  employed  primarily  as  a  measure  of  the 
degree  of  user  acceptance  of  a  new  technology. 

Information  system  characteristics  affect  the  efficiency  and  effectiveness  of  the 
user's  interaction  with  a  computer  system.  Some  of  the  characteristics  which  have 
been  investigated  include  response  time  (Schewe,  1976;  Fuerst  and  Cheney,  1982), 
accuracy  and  relevancy  of  output  (Schewe,  1976;  Srinivasan,  1985;  Fuerst  and 
Cheney,  1982;  O'Reilly,  1982),  stability  and  security  (Srinivasan,  1985), 
presentation  format  (Srinivasan,  1985;  Fuerst  and  Cheney,  1982;  Ein-Dor,  Segev, 
and  Steinfeld,  1982),  sophistication  of  DSS  model  (Henderson  and  Schilling,  1985), 
and  user  interface  (Fuerst  and  Cheney,  1982;  Raymond,  1985).  Many  of  these  latter 
studies  have  an  ergonomic  orientation  in  which  system  use  is  employed  to  assess  the 
efficiency  of  the  man-machine  design. 

Individual  differences  affect  beliefs,  which  in  turn  affect  attitudes,  intentions,  and 
information  system  utilization.  Examples  of  individual  characteristics  which  have 
been  studied  include  age  (Fuerst  and  Cheney,  1982;  McCosh,  1984;  Culnan,  1983), 
experience  (Fuerst  and  Cheney,  1982;  McCosh,  1984;  Culnan,  1983;  O'Reilly,  1982; 
Alavi  and  Henderson,  1981),  educational  level  (Fuerst  and  Cheney,  1982;  O'Reilly, 
1982),  and  cognitive  style  (Fuerst  and  Cheney,  1982).  In  the  context  of  individual 
differences  studies,  there  is  usually  an  implicit  assumption  that  utilization  is  an 
indicator  of  MIS  success  (Zmud,  1979). 

Task  characteristics  refer  to  the  nature  of  the  tasks  users  must  execute.  This  type  of 
independent  variable  was  considered  less  often  in  utilization  research  than  the 
preceding  three.  The  primary  task  characteristics  which  have  been  investigated  are 
complexity  and  uncertainty  (McCosh,  1984;  Culnan,  1983;  O'Reilly,  1982;  Mann  and 
Watson,  1984).  In  these  types  of  studies,  utilization  is  employed  as  a  measure  of  the 
suitability  of  the  information  system  to  the  needs  of  different  types  of  users. 

The  purpose  of  a  study  was  associated  with  both  the  nature  of  the  task  under  study 
and  the  means  of  collecting  utilization  information.  For  example,  if  the  activity 
under  study  was  a  defined  task,  such  as  editing  a  file  or  writing  a  query,  then  the 
objective  of  the  exercise  was  usually  to  minimize  effort  through  better  design  of  the 


technology.  If  it  was  an  undefined  or  unspecified  set  of  tasks  for  which  the  system 
was  used,  then  more  use  was  usually  assumed  to  be  better.  More  use  is  taken  to 
indicate  the  user's  belief  that  the  system  is  beneficial.  Less  predictably,  if  the  means 
of  gathering  utilization  information  was  unobtrusive,  such  as  collecting  machine 
usage  statistics,  different  independent  variables  were  studied  than  if  the  utilization 
was  operationalized  using  self-reported  data.  These  categories  of  research  are 
briefly  reviewed  below. 

(1)  DEFINED  TASK,  UNOBTRUSIVE  OPERATIONALIZATION 

The  goal  of  this  type  of  research  is  almost  always  to  suggest  design  choices  which 
will  optimize  the  efficiency  of  the  use  of  the  information  system.  For  example,  many 
studies  have  been  performed  to  determine  whether  users  can  execute  a  defined  task 
more  quickly  using  menus  rather  than  a  command  language,  or  a  mouse  rather  than 
a  keyboard.  Note  that  this  type  of  research  is  distinct  from  studies  which  examine 
the  effect  of  presentation  format  or  other  system  characteristics  on  individual 
performance  (for  example,  see  Ives  ( 1982),  and  DeSanctis  (1984)), 

Since  the  literature  in  this  area  is  extensive  and  quite  separate  from  the  main  body 
of  MIS  literature,  we  will  not  attempt  to  survey  it  here.  However,  it  is  simple  to 
describe  its  general  structure.  The  typical  study  is  a  laboratory  experiment 
comparing  two  systems  with  different  user  interface  characteristics.  The  unit  of 
measurement  employed  is  usually  a  machine  usage  statistic  such  as  a  keystroke  or 
carriage  return.  Sometimes  it  is  elapsed  time.  The  independent  variable  is  always 
some  sort  of  system  characteristic,  such  as  type  of  query  language  used. 

(2)  DEFINED  TASK,  SELF-REPORT  OPERATIONALIZATION 

Research  in  this  area  would  presumably  have  the  same  goals  as  the  studies  in  (1). 
However,  no  studies  were  found  in  this  area,  possibly  because  self-reported  measures 
of  use  would  not  be  accurate  enough  to  covary  significantly  with  the  feature  of 
information  system  design  under  study.  By  using  objective  machine  usage  statistics, 
most  of  the  attenuating  efi'ects  of  measurement  error  and  bias  can  be  eliminated. 

(3)  UNDEFINED  TASK,  UNOBTRUSIVE  OPERATIONALIZATION 

All  four  studies  found  in  this  area  were  quasi-experiments  which  attempted  to 
establish  links  between  individual  attitudes  and  utilization.    The  results  of  the 


studies  were  somewhat  mutually  conflicting  (see  Appendix).  This  is  not  surprising, 
in  view  of  the  fact  that  each  of  the  four  researchers  employed  a  different  set  of 
measurement  units.  Robey  (1979)  used  the  percentage  of  records  updated.  Ginzberg 

(1981)  employed  connect  time  and  number  and  frequency  of  computer  sessions. 
Schewe  (1976)  used  additional  monthly  requests  for  information  by  managers,  and 
Srinivasan  (1985)  employed  number  of  accesses  per  month,  connect  time,  and 
number  of  light,  average,  and  heavy  users.  The  range  of  different  measurements 
observed  is  probably  partly  a  consequence  of  the  lack  of  both  a  cumulative  tradition 
and  a  generally  accepted  definition  of  utilization  in  this  body  of  literature. 

(4)  UNDEFINED  TASK,  SELFREPORT  OPERATIONALIZATION 

Very  broadly,  the  literature  in  this  area  attempted  to  link  implementation 
characteristics  and  individual  differences  to  the  degree  of  I/S  utilization.  The 
results  have  been  rather  fragmented,  with  many  different  measures  being  used  and 
many  different  theories  being  tested.  For  example,  O'Reilly  (1982)  found  that 
accessibility  of  sources  of  information  primarily  determines  utilization;  Fuerst  and 
Cheney  (1982)  found  that  user  training,  accuracy  and  relevancy  of  output,  and 
experience  are  the  relevant  factors;  McCosh  (1984)  determined  that  the  level  of 
qualifications  held  by  the  superior  of  the  main  user  is  the  strongest  determinant;  and 
Raymond  (1985)  found  a  firm's  EDP  experience  to  have  strong  explanatory  power. 
Again,  a  possible  explanation  for  this  fragmentation  lies  in  the  absence  of  a 
cumulative  tradition  in  the  literature. 

The  predominant  methodologies  employed  in  this  area  were  quasi-experiments  and 
informal  surveys.  A  few  studies  were  lab  experiments  or  case  based.  In  general, 
questionnaires  and  interviews  were  used  for  data  collection.  The  unit  of 
measurement  used  varied  widely.  Fuerst  and  Cheney  (1982)  employed  a  series  of 
scale  questions  concerning  general  and  specific  use.  Ein-Dor,  Segev,  and  Steinfeld 

(1982)  used  past  use  and  intended  future  use  of  a  PERT  system.  McCosh  (1984)  used 
a  five-point  scale  estimate  of  system  success,  with  a  score  of  two  implying  use  but 
ultimate  failure.  Culnan  (1983)  employed  a  series  of  scale  questions  concerning  end 
user  access.  Raymond  (1985)  used  a  series  of  scale  questions  concerning  utilization  in 
general.  O'Reilly  (1982)  employed  scale  questions  pertaining  to  frequency  of  use. 
Two  lab  experiments  opera tionalized  use  as  a  binary  variable. 


DISCUSSION 

Two  features  stand  out  from  this  summary  of  the  bulk  of  the  last  ten  years  of 
utilization  literature.  One  is  the  lack  of  an  accumulation  of  knowledge  in  this  area, 
which  is  in  part  attributable  to  the  lack  of  any  standardized  measures.  This  problem 
can  be  traced  back  to  a  lack  of  underlying  theory  to  guide  the  choice  of  measures. 
The  other  is  the  relatively  large  proportion  of  studies  which  employed  self-reported 
utilization  measures,  even  though  unobtrusive  measures  are  often  obtainable  and, 
as  a  rule,  more  accurate. 

In  the  absence  of  an  underlying  theory,  the  research  methodology  rather  than  the 
theory  tends  to  drive  the  choice  of  utilization  measure,  which  is  inappropriate.  This 
problem  is  exacerbated  when  a  single  utilization  measure  is  used  in  studies 
involving  multiple  independent  variable  types.  As  will  be  discussed  later, 
consistency  of  measurement,  and  thus  a  cumulative  tradition,  can  only  be  achieved  if 
the  proper  reference  theory,  rather  than  the  research  methodology,  guides 
utilization  definition  and  measurement.  In  the  next  section  we  will  discuss  several 
relevant  reference  disciplines  in  detail. 

Unobtrusive  utilization  measures  have  been  employed  infrequently  for  several 
reasons.  One  is  that  machine  usage  statistics  are  often  more  difficult  to  obtain  from 
organizations  than  completed  questionnaires  or  interviews.  If  more  sophisticated 
usage  statistics  are  desired,  it  may  even  be  necessary  to  alter  the  characteristics 
and/or  performance  of  the  information  system  extensively.  Another  reason  is  that 
often  researchers  can  only  collect  utilization  data  some  time  after  the  utilization  has 
taken  place.  Unfortunately,  post  hoc  self-report  data  tends  to  be  weaker  and  less 
accurate  than  unobtrusive  data. 

IV.  Reference  Theories  for  Utilization 

Measurement 

All  the  studies  we  have  reviewed  have  a  similar  structure;  some  aspects  of  system 
use  is  hypothesized  to  be  affected  by  some  other  aspects  of  the  design  and 
implementation  process,  or  by  characteristics  of  the  information  system,  the  task, 
the  individual  user  or  their  interaction.  But  what  aspects  of  system  use  are  most 
appropriate  to  measure?    This  depends  in  large  part  upon  which  independent 
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variable  is  under  study.  Different  independent  variables  will  alter  utilization  in 
different  ways.  Utilization  needs  to  be  defined  and  operationalized  in  terms  that  can 
measure  these  effects  best.  This  suggests  that  it  is  the  independent  variables 
employed  in  utilization  studies  which  should  determine  the  choice  of  reference 
theory.  In  this  section  we  turn  to  a  brief  discussion  of  some  theories  that  correspond 
to  different  types  of  independent  variables  employed  in  utilization  research  and  the 
implications  of  these  theories  for  utilization  measurement. 

A  review  of  the  literature  revealed  three  reference  theories  that  are  useful  for 
linking  various  types  of  independent  variables  with  utilization.  In  the  case  of 
implementation  variables,  Keen  (1981)  argued  that  the  Lewin-Schein  (Schein,  1969) 
model  of  change  management  underlies  much  of  the  MIS  implementation  literature. 
In  the  case  of  individual  differences  and  information  systems  variables,  two  other 
theories  are  suggested  by  Zmud's  (1979)  model  linking  individual  differences  to  MIS 
success.  One  theory,  linking  individual  differences  with  MIS  user  attitudes  and 
involvement,  is  Fishbein's  (1979)  theory  of  reasoned  action.  The  other,  which  links 
individual  cognitive  differences  to  desirable  MIS  design  characteristics,  is  the  theory 
of  ergonomics. 

In  summary,  the  three  theories  (and  the  different  determinants  of  utilization  they 
explain)  which  we  will  review  are  as  follows:  the  Lewin-Schein  model  of  change 
management  (implementation  variables),  the  Fishbein  theory  of  reasoned  action 
(individual  differences  variables),  and  ergonomic  theories  of  man-machine 
interaction  (individual  differences  and  information  systems  variables).  These  three 
reference  theories  underlie  a  great  deal  of  the  utilization  research.  Since  task 
characteristics  have  received  so  little  attention  in  the  utilization  literature,  we  will 
not  discuss  any  reference  theories  corresponding  to  that  area.  The  primary  goal  of 
this  discussion  is  to  identify  the  relevant  utilization  definitions  each  reference 
theory  suggests.  A  secondary  goal  is  to  comment  on  how  the  utilization  definitions 
are  linked  to  performance. 

(1)  IMPLEMENTATION  VARIABLES- LEWIN-SCHEIN  MODELOF 
CHANGE  MANAGEMENT 

The  Lewin-Schein  (1969)  model  of  implementation  and  Kolb  and  Frohman's  (1970) 
extension  of  it  constitute  the  implicit  theory  used  in  much  of  the  MIS 
implementation  literature.  A  related  theory  (Rogers,  1962)  of  acceptance  of 


innovations  parallels  the  Lewin-Schein  framework.  In  these  models, 
implementation  is  viewed  as  a  three-stage  process.  In  the  first  stage,  the 
organizational  environment  is  "unfrozen".  By  this  we  mean  a  climate  and  contract 
for  a  change  in  the  environment  (in  this  case,  an  MIS  implementation)  is  created. 
For  example,  a  climate  for  change  exists  when  users  are  made  to  feel  that  the 
organization  needs  an  MIS  in  order  to  improve  performance.  In  the  second  stage,  the 
change  is  actually  implemented.  In  the  third  stage,  the  change  is  institutionalized, 
that  is,  it  actually  becomes  an  integral  part  of  the  organization. 

These  models  equate  implementation  success  with  the  degree  to  which  the  MIS  is 
institutionalized  in  the  organization.  Utilization  is  often  used  as  a  surrogate 
measure  of  the  degree  of  institutionalization  -  the  more  a  system  is  used  the  more  it 
becomes  an  integral  part  of  the  organization.  There  are  at  least  three  aspects  of 
system  use  that  are  most  relevant  to  the  institutionalization  construct.  One  is  the 
degree  to  which  users  are  dependent  upon  the  MIS  after  it  has  been  implemented. 
That  is,  if  the  MIS  were  suddenly  to  disappear,  would  its  absence  be  missed  by  the 
organization?  Another  is  the  extent  to  which  users  feel  an  ownership  for  the  MIS.  Do 
they  feel  that  they  have  control  of  the  system,  or  does  the  MIS  department  dictate 
how  it  is  used?  A  third  is  the  degree  to  which  use  of  the  MIS  is  routinized,  that  is, 
used  as  part  of  the  standard  operating  procedure  of  the  organization.  These  are  the 
three  aspects  of  information  system  utilization  most  relevant  to  implementation 
research. 

Note  that  the  concept  of  institutionalization  is  very  different  from  the  concept  of 
performance.  Indeed,  institutionalization  of  an  MIS  can  occur  without  there  being 
any  performance  change  observed  in  the  organization.  We  will  explore  the  link  to 
performance  in  greater  detail  in  Section  V. 

(2)  INDIVIDUAL  DIFFERENCES  -  FISHBEIN'S  THEORY  OF  REASONED 
ACTION 

The  theory  of  reasoned  action  is  a  model  widely  employed  in  research  that  links 
beliefs  and  attitudes  to  behavior.  It  is  illustrated  in  Figure  2.  The  theory  suggests 
that  the  use  of  an  information  system  is  best  predicted  by  an  individual's  intention  to 
use  the  system.  This  intention  is  determined  by  some  weighted  combination  of  the 
individual's  attitude  toward  using  the  system  and  his  or  her  subjective  assessment 
of  the  social  acceptability  of  this  behavior.  Attitude  toward  use  is  in  turn  determined 
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by  the  individual's  beliefs  about  the  consequences  of  using  the  system  and  social 
acceptability  is  determined  by  assessing  the  opinions  of  some  referent  group. 


The  person's  beliefs 
that  the  behavior 
leads  to  certain 
outcomes  and  his 
evaluations  of 
these  outcomes 


Attitude  toward 
the  behavior 


The  person's  beliefs 
that  specific  individuals 
or  groups  think  he 
should  or  should  not 
perform  the  behavior 
and  his  motivation  to 
comply  with  the 
specific  referents 


Relative  importance 
of  attitudinal  and 
normative 
considerations 


Subjective  norm 


Intention 


Behavior 


Source:Fishbein(1979) 


Figure  2.  Theory  of  Reasoned  Action 

Thus,  according  to  this  theory,  whether  or  not  an  individual  uses  an  information 
system  can  be  traced  back  to  his  or  her  beliefs  about  the  benefits  that  will  derive 
from  its  use.  Individual  differences,  such  as  age,  computer  experience,  or 
educational  background  affect  these  beliefs  which  in  turn  affect  whether  or  not  a 
system  is  used  through  attitudes,  norms,  and  intentions.  Here  again  the  reference 
theory  can  shape  our  understanding  of  what  aspects  of  system  utilization  are 
important  to  measure.  In  applying  this  particular  theory,  we  would  want  to  define 
utilization  as  the  active  use  of  the  system  or  some  aspect  of  the  system  and  to  model 
it  as  a  binary  variable,  use  and  non-use  of  the  system.  This  definition  of  utilization  is 
very  different  from  the  definition  that  is  directly  relevant  to  implementation 
research.  There  is  no  direct  relationship  between  use,  as  defined  through  the  theory 
of  reasoned  action,  and  performance.  Whether  use  or  non-use  of  a  system  results  in 
performance  improvements  depends  upon  the  business  context  and  a  myriad  of  other 
intervening  variables. 

(3)  INDIVIDUAL  DIFFERENCES  AND  INFORMATION  SYSTEMS 
CHARACTERISTICS  -  ERGONOMICS 

Broadly  stated,  one  of  the  goals  of  ergonomics  is  to  provide  for  efficient  man-machine 
interfaces  that  are  suited  to  the  physical  and  cognitive  capabilities  of  man.  Thus 
ergonomics  has  two  related  goals,  to  reduce  human  physical  effort  through  eflicient 
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design  and  to  reduce  cognitive  effort  through  designs  that  are  easy  to  use  and  robust. 
Attainment  of  both  these  goals  can  be  unobtrusively  measured  with  utilization 
variables. 

In  information  systems  design  research,  more  attention  has  been  focused  upon 
reducing  physical  effort  than  cognitive  effort,  perhaps  because  the  associated 
utilization  measures  are  easier  to  obtain.  Measures  of  physical  effort  include  such 
constructs  as  keystrokes,  carriage  returns,  and  elapsed  time  for  a  fixed  task. 
Measures  of  cognitive  effort  are  often  difficult  to  obtain,  so  surrogates  such  as  error 
rates  and  requests  for  help  are  often  used. 

The  link  between  utilization  in  the  sense  of  physical  and  cognitive  effort  and 
performance  is  relatively  straightforward;  minimizing  effort  expended  to  execute  a 
fixed,  defined  task  improves  performance  because  it  saves  time.  This  allows  the  user 
to  spend  more  time  engaged  in  other  activities. 

V.  Conclusions:  Needs  for  Future  Research 

Having  reviewed  the  utilization  literature  and  discussed  three  relevant  reference 
theories,  we  now  turn  to  recommendations  for  future  research. 

UTILIZATION  AS  A  DEPENDENT  VARIABLE:  DEFINITION  AND 
MEASUREMENT 

The  research  we  have  surveyed  indicates  that  linkages  between  utilization  and  its 
determinants  are  not  well  understood.  As  Section  HI  showed,  the  literature  is 
somewhat  fragmented,  and  in  some  cases  conflicting  results  have  been  obtained. 
The  lack  of  theoretical  understanding  has  in  turn  caused  methodological  problems. 
Many  utilization  studies  have  measured  the  relationship  between  various 
independent  variables  and  utilization  directly  without  paying  attention  to 
intervening  variables.  For  example,  several  studies  have  investigated  the 
relationship  between  individual  differences  and  utilization  without  controlling  for 
any  intervening  attitudinal  variables,  thereby  limiting  the  strength  of  the  results. 

How  can  the  determinants  of  utilization  be  better  understood?  The  first  step  is  to 
recognize  that  utilization  is  a  multidimensional  variable,  and  that  different 
definitions  of  utilization  will  apply  depending  on  the  process  under  study,  whether  it 
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be  implementation,  reasoned  action,  man-machine  interaction,  or  some  other 
process.  If  the  process  to  be  studied  has  been  determined,  the  second  step  is  to  use 
the  applicable  reference  theory  to  guide  research  design.  Reference  theories  can  aid 
both  in  identifying  appropriate  independent  and  intervening  variables  to  relate  to 
utilization  and  in  providing  definitions  of  utilization  which  will  suggest  good 
opera  tionalizations. 

As  important  as  the  chosen  definition  of  utilization  is  the  definition  of  information 
technology  which  is  chosen.  Here  again,  it  is  important  to  select  a  definition  which  is 
appropriate  for  the  theory.  For  example,  from  the  perspective  of  the  theory  of 
reasoned  action,  information  technology  can  be  characterized  as  a  system  which 
provides  potentially  useful  functions.  This  is  because  the  theory  predicts  that  it  is 
beliefs  about  the  consequences  of  using  the  system's  functions  which  ultimately 
determines  utilization. 

One  consequence  of  the  lack  of  consistency  in  utilization  definitions  in  MIS  research 
is  the  lack  of  consistency  of  utilization  measures  as  well.  It  has  been  pointed  out  that 
this  lack  of  measurement  consistency  often  makes  it  difficult  to  compare  different 
studies  in  this  area.  One  way  to  alleviate  this  problem  is  to  adopt  standardized 
utilization  measures.  However,  adopting  standardized  measures  would  require  a 
standardized  definition  of  utilization  as  well.  This  is  inappropriate,  since  as 
discussed  above,  utilization  is  process  dependent.  The  best  we  could  hope  for  would 
be  to  adopt  standardized  measures  within  the  context  of  a  single  theory  based  on  the 
definition  of  utilization  the  reference  theory  suggests.  For  example,  an  instrument 
based  on  the  Lewin-Schein  framework  could  be  developed  which  would  measure  user 
dependency  and  ownership  towards  an  MIS. 

A  second  issue  concerns  the  use  of  objective  utilization  measures.  Even  though 
machine  usage  statistics  are  routinely  logged  and  readily  accessible  (at  least  in  the 
case  of  mainframe  computer  systems),  thus  far  they  have  been  employed  far  less 
frequently  than  self-reported  measurements  in  the  literature.  It  is  recommended 
that  researchers  use  machine  usage  measures  rather  than  the  more  subjective  self- 
reported  measures  whenever  possible,  since  as  a  rule  objective  measures  are  more 
accurate.  Machine  usage  measures  can  be  made  even  more  accurate  by  imposing 
controls.  For  example,  if  connect  time  is  the  measure  being  used,  users  could  be 
logged  off  automatically  during  extended  idle  periods. 
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Care  must  be  exercised  in  the  selection  of  objective  machine  usage  measures, 
however.  For  example,  if  effort  is  the  construct  being  measured,  it  is  not  appropriate 
to  use  the  number  of  information  products  such  as  reports  as  the  operationalization. 
The  reason  is  that  information  products  are  in  fact  a  function  of  effort  and  the 
characteristics  of  the  information  system  rather  than  a  measure  of  the  effort  itself. 

UTILIZATION  AS  AN  INTERVENING  VARIABLE 

Although  this  paper  has  focused  on  utilization  as  a  dependent  variable,  it  is 
appropriate  to  examine  the  role  of  utilization  in  a  broader  context  briefly.  Doing  so 
will  serve  to  place  the  employment  of  utilization  as  a  dependent  variable  in 
perspective,  as  well  as  to  suggest  additional  avenues  for  future  research. 

In  a  theory  linking  information  technology  and  performance,  utilization  can  be 
viewed  as  an  intervening  variable  (see  Figure  3).  That  is,  utilization  is  partially 
determined  by  information  technology  variables,  and  is  also  one  of  the  many 
variables  which  ultimately  affects  performance.  We  will  refer  to  theoretical 
relationships  between  information  technology  and  utilization  as  backward  linkages, 
and  between  utilization  and  performance  as  forward  linkages.  As  Figure  3  shows, 
neither  backward  or  forward  linkages  are  necessarily  direct.  For  example,  according 
to  the  theory  of  reasoned  action,  utilization  is  determined  by  an  individual's 
intention  to  use  a  system,  rather  than  directly  by  the  availability  of  the  technology. 
Similarly,  one  could  argue  that  utilization  affects  performance  by  means  of 
organizational  structural  changes.  This  paper  has  so  far  addressed  itself  exclusively 
to  improving  our  understanding  of  backward  linkages;  possible  steps  towards  a 
better  understanding  of  forward  linkages  are  discussed  below. 

It  is  clear  that  forward  linkages  must  exist  if  a  system  is  to  affect  performance,  since 
information  technology  cannot  have  an  impact  on  performance  if  it  is  not  used  in 
some  way.  However,  the  nature  of  these  linkages  is  not  at  all  clear.  There  are  cases 
in  which  increased  utilization  actually  leads  to  a  degradation  in  performance,  for 
example,  when  there  is  a  fixed  task  to  perform  and  the  system  is  designed 
inefilciently  such  that  it  takes  more  effort  to  complete  the  task  than  is  necessary,  or 
the  system  is  so  personally  desirable  to  users  that  they  expend  considerable  effort 
using  the  system  in  nonproductive  ways.  Consequently,  utilization  alone  is  not 
sufficient  to  predict  performance  accurately,  which  means  that  forward  linkages  are 
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Variables 
Affecting 
Utilization 

Other 
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performance 

Information 
Technology 

"        fc 

Utilization 
of  System 

" 

Performance 

Figure  3.  Utilization  as  an  Intervening  Variable 


bound  to  be  complex.  Thus,  utilization  is  a  necessary  but  insufficient  condition  for  a 
system  to  affect  performance. 

This  complexity  of  forward  linkages  has  an  important  implication  for  utilization 
research.  Some  of  the  studies  performed  to  date  have  assumed  that  utilization  of  an 
information  system  is  a  surrogate  measure  of  its  effectiveness  or  success.  The 
preceding  discussion  suggests  that  to  achieve  a  better  understanding  of  the  link 
between  information  technology  and  performance,  it  would  be  fruitful  to  view 
utilization  as  an  independent  variable  affecting  performance,  rather  than  an 
indicator  of  performance  itself  Employment  of  utilization  as  a  dependent  variable 
should  be  restricted  to  the  context  of  a  reference  theory  such  as  one  of  those 
previously  discussed. 

If  utilization  is  treated  as  an  independent  variable  affecting  performance,  then  there 
are  at  least  two  ways  in  which  it  can  be  defined  and  measured.  One  is  to  identify  the 
components  of  performance  which  are  of  interest  and  measure  the  facets  of 
utilization  which  correspond  to  these  components.  For  example,  if  we  are  interested 
in  the  effect  of  information  technology  on  communication,  then  clearly  we  should 
focus  on  utilization  of  the  communications-oriented  features  of  a  system  as  opposed 
to  utilization  of  some  other  information  systems  function.  Another  is  to  use  the 
theoretical   factors   which    affect   performance    to   suggest   appropriate 
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operationalizations.  These  techniques  can  also  be  used  in  the  case  where  the 
ultimate  dependent  variable  is  something  other  than  performance,  such  as 
organizational  structural  change. 

It  should  be  noted  that  the  methods  for  measuring  utilization  as  an  independent 
variable  are  sometimes  completely  different  from  those  used  for  measuring  it  as  an 
dep>endent  variable.  For  example,  to  measure  the  effect  of  utilization  on  individual 
performance,  it  may  be  useful  to  measure  utilization  in  terms  of  the  distribution  of 
computer  time  spent  performing  various  functions  such  as  spreadsheet  and 
electronic  mail,  but  it  would  not  be  useful  to  use  this  particular  measure  in  the 
context  of  any  of  the  reference  theories  discussed  above.  This  is  a  consequence  of  the 
difficulty  of  developing  a  simple  and  accurate  theory  linking  information  technology 
and  performance. 

Although  understanding  the  nature  of  both  backward  and  forward  linkages  is  an 
important  step,  taken  together  the  two  linkages  do  not  provide  a  complete  and 
consistent  view  of  the  significance  of  utilization  as  an  intervening  variable. 
Constructing  a  continuous  theoretical  path  from  information  technology  through 
utilization  to  performance  may  still  be  difilcult.  Operational  definitions  of 
utilization  obtained  from  performance  components  will  not  necessarily  correspond  to 
those  obtained  through  identifying  utilization  determinants. 

In  summary,  utilization  is  an  important  intervening  variable  in  the  link  between 
information  technology  and  performance.  Although  much  valuable  research  has 
been  performed,  utilization  is  still  not  well  understood  or  well  measured.  Drawing 
upon  reference  theories  for  definitions  and  operationalizations  will  move  the  MIS 
field  closer  to  to  achieving  this.  It  is  difficult  to  trace  a  clean  theoretical  path 
between  information  technology  and  performance  with  utilization,  but  impossible  to 
trace  such  a  path  without  it. 
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